Low Bandwidth Video Conversation Using Anatomical Renconstruction of Facial Expressions over the Internet

As the Internet becomes fast and ubiquitous, the demand for realistic multimodal communication will increase substantially for human-like human computer interaction and video conversation over the Internet. This demand will easily undo the bandwidth gain of the Internet2. This will necessitate realistic image visualization and rendering at the client end using a low bandwidth textual description of dynamic changes in multimodal interaction. In this model we describe an integration of a client end detailed anatomical model of facial expressions superimposed on a single high resolution image of a speaker. The facial expression rendering changes dynamically based upon textual information about emotional expression transmitted over the Internet. This paper describes FEML (Facial Expression Markup Language) ⎯ an XML based language for 3D dynamic anatomical modeling of human face expression over the Internet and its integration with the corresponding sound transmitted over the Internet. FEML has two parts: the user level that encodes the emotions and the corresponding audio, and the system level macros for anatomical muscle modeling of facial expression. The performance evaluation shows that for longer conversation the efficiency achieved is significant to frame based audio-visual transmission formats such as MPEG movies.